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Report  No.  97-050  December  17,  1996 

MEMORANDUM  FOR  DIRECTOR,  DEFENSE  FINANCE  AND  ACCOUNTING 

SERVICE 


SUBJECT:  Evaluation  of  Controls  Over  Workflow  Applications  Selected  for 
Electronic  Document  Management  (Project  No.  6FG-5019.00) 


Introduction 

We  are  providing  this  evaluation  report  for  your  information  and  use.  This 
report  is  one  in  a  series  of  reports  on  the  Electronic  Document  Management 
(EDM)  system.  The  Director,  Defense  Finance  and  Accounting  Service 
(DFAS),  requested  assistance  from  the  Inspector  General,  DoD,  in  reviewing 
the  design  and  development  of  the  EDM  system  to  ensure  that  management  and 
system  control  requirements  are  met  before  DFAS  accepts  the  EDM  system. 
We  reviewed  the  EDM  system  in  a  series  of  design  reviews  and  tested  the  EDM 
system  at  the  DFAS  Omaha  Operating  Location  (OPLOC)  during  system 
acceptance  training  and  testing.  We  briefed  the  Deputy  Director,  Plans  and 
Management,  DFAS,  on  July  11,  1996. 


Evaluation  Results 

We  commend  DFAS  for  its  critical  examination  of  the  EDM  system  during  the 
testing  period,  and  for  a  detailed  accounting  of  performance  and  control 
deficiencies  identified  by  the  EDM  system  acceptance  team.  The  EDM  system 
acceptance  test  showed  that  controls  over  the  EDM  vendor  payment  process  and 
workflows  can  achieve  management  control  objectives  related  to  the 
completeness,  accuracy,  and  authorization  of  data.  However,  improvements  are 
needed  in  the  security  controls  over  EDM  system  data.  Specifically,  controls 
were  needed  to  limit  log-on  attempts,  meet  auditability  requirements,  and 
protect  secure  files.  DFAS  initiated  corrective  action;  therefore,  we  are  not 
making  any  recommendations  at  this  time. 


Objectives 

The  overall  objective  of  the  evaluation  was  to  determine  whether  the  EDM 
system  can  achieve  management  control  objectives  related  to  the  completeness, 
accuracy,  and  authorization  of  data,  and  whether  the  system  can  meet 
requirements  for  document  retention.  Specifically,  we  determined  whether 
controls  over  workflow  applications  selected  for  EDM  were  adequate.  During 
this  phase  of  the  evaluation,  we  did  not  review  procedures  for  rapid  application 
development  and  phased  implementation  to  assess  the  future  auditability  of 
imaging  systems. 


Scope  and  Methodology 

Scope  and  Methodology.  The  scope  of  our  evaluation  was  the  EDM  system 
prototype  scheduled  for  use  in  making  vendor  payments  at  the  Omaha  OPLOC. 
To  achieve  our  objectives,  we  participated  in  reviews  of  the  EDM  system's 
design,  training,  and  acceptance  testing.  Enclosure  1  lists  the  DoD 
organizations  that  participated  in  the  EDM  system  acceptance  testing  and  the 
organizations  we  visited  or  contacted.  Before  testing,  we  received  formal 
training  on  the  EDM  system  and  assisted  the  functional  and  technical  team 
leaders  in  developing  a  test  plan.  During  testing,  we  assisted  management  in 
evaluating  system  controls  and  security  by  testing  the  system  and  assessing 
problem  reports.  We  maintained  our  independence  by  acting  in  an  advisoiy 
capacity  only.  We  did  not  work  with  the  system  design  and  development  teams 
or  participate  in  the  DFAS  decisionmaking  process. 

Technical  Support.  The  Quantitative  Methods  Division  of  the  Analysis, 
Planning,  and  Technical  Support  Directorate,  Office  of  the  Inspector  General 
for  Auditing,  DoD,  provided  technical  support  in  testing  the  EDM  UNIX 
operating  system  security,  reviewing  evaluation  steps  for  technical  accuracy  and 
effectiveness,  and  reviewing  the  Omaha  OPLOC  statistical  plan  for  selecting  the 
voucher  packets  used  in  EDM  system  acceptance  testing. 


Evaluation  Period  and  Standards.  This  financial-related  evaluation  was 
performed  from  November  1995  through  August  1996  in  accordance  with 
standards  implemented  by  the  Inspector  General,  DoD.  We  announced  tins 
evaluation  on  October  31,  1995,  and  performed  fieldwork  at  i die  Integrated  Test 
Facility  Camp  Hill,  Pennsylvania,  and  the  Omaha  OPLOC.  We  also 
performed  fieldwork  at  the  Omaha  OPLOC  during  EDM  system  acceptance 
training  and  testing  from  April  1,  1996,  to  May  28,  1996.  We  completed  our 
fieldwork  on  August  9,  1996.  In  performing  this  evaluation,  we  did  not  rely  on 
computer-processed  data. 


Scope  Limitations.  We  identified  the  requirements  for  retention  of 
electronically  stored  documents,  but  we  were  unable  to  conduct  tests  to 
determine  whether  the  EDM  system  could  meet  those  requirements.  This  was 
because  the  DFAS  EDM  Project  Manager  and  the  system  developer  did  not  plan 
to  convert  document  images  to  optical  disks  as  part  of  EDM  system  acceptance 
testing.  The  DFAS  Financial  Systems  Organization  plans  to  test  document 
storage  and  retrieval  from  optical  disks  before  installing  the  optical  long-term 
storage  subsystem.  We  did  not  review  rapid  application  development 
procedures  because,  according  to  the  system  developer,  those  procedures  were 
not  followed.  Also,  we  did  not  review  the  phased  implementation  strategies 
because  evaluating  the  EDM  system  had  higher  priority.  We  will  review  the 
procedures  for  rapid  application  development  and  phased  implementation  during 
the  next  phase  of  our  evaluation  of  the  EDM  system. 


Prior  Audits  and  Other  Reviews 

No  audit  reports  have  been  issued  in  the  last  5  years  concerning  system  controls 
over  EDM  systems  in  DoD,  and  no  reviews  have  been  performed. 
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Background 

In  1995,  DFAS  selected  the  Omaha  OPLOC  for  the  design,  development,  and 
implementation  of  the  EDM  system  prototype  for  vendor  payments.  Also  in 
1995,  the  system  developer  conducted  a  business  process  analysis  of  the  Omaha 
OPLOC,  which  showed  that  four  functional  areas  and  associated  DFAS  business 
processes  would  benefit  from  implementing  an  EDM  system.  The  four  areas 
selected  were  vendor  payments,  disbursing,  accounting  operations,  and  travel 
accounting.  Within  those  4  areas,  27  workflow  processes  were  identified  and 
selected  for  implementation.  Workflow  is  the  automation  of  existing  business 
procedures  that  control  how  work  travels  through  an  organization.  Initially, 
7  workflows  associated  with  invoice  payments  (representing  80  percent  of  the 
total  work  load)  would  be  implemented;  the  other  20  workflows  would  be 
implemented  later. 

OPLOC  Consolidation  and  Work  Load.  As  of  July  31,  1996,  the  Omaha 
OPLOC  had  consolidated  22  Defense  Accounting  Offices  for  7  Air  Force  and 
15  Air  National  Guard  bases.  The  vendor  payment  process  at  the  Omaha 
OPLOC  is  document-intensive,  and  volume  has  significantly  increased  as  the 
result  of  DFAS  consolidation.  DFAS  projected  that  during  1996,  the  work  load 
would  increase  from  12,000  to  25,500  vendor  payments  monthly,  creating 
approximately  765,000  pages  of  documentation  each  month  by  December  1996. 
DFAS  expects  a  significant  increase  in  productivity  resulting  from  EDM 
implementation.  For  example,  economic  analysis  by  DFAS  shows  that  for 
processing  payments  on  invoices,  cycle  time  will  improve  by  41  percent  and 
direct  labor  costs  will  improve  by  43  percent. 

EDM  System  Design.  The  EDM  system  is  designed  to  replace  paper 
documents  with  electronically  imaged  documents,  control  the  workflow  through 
the  OPLOC  vendor  payments  section,  and  match  obligation  documents, 
invoices,  and  receiving  reports  to  payment  vouchers  created  by  the  Integrated 
Accounts  Payable  System  (IAPS).  The  EDM  system  is  an  automated 
information  system  that  consists  of  three  subsystems:  document  capture, 
indexing,  and  workflow.  Document  capture  is  the  scanning  of  paper  documents 
to  create  computer  images  of  those  documents.  The  scanner  automatically  tags 
the  paper  document  and  image  with  a  document  identifier,  allowing  retrieval  of 
the  paper  document  when  necessary.  Next,  the  indexer  uniquely  marks  the 
image  by  document  type  and  categorizes  the  image  according  to  a  predetermined 
workflow,  which  allows  subsequent  retrieval,  viewing,  and  processing. 
Workflows  consist  of  a  series  of  tasks  to  be  accomplished  in  processing 
obligations,  invoices,  receiving  reports,  vouchers,  and  various  combinations  of 
those  documents. 

The  EDM  system  consists  of  hardware  and  software,  and  uses  the  local  area 
network  at  the  OPLOC.  The  hardware  consists  of  scanners  that  convert  paper 
documents  to  electronic  images;  servers  that  run  the  document  scanning 
programs,  maintain  the  operating  systems,  and  execute  the  main  indexing 
program  and  workflow  programs;  direct  access  storage  devices  that  provide  data 
access  and  storage;  an  optical  disk  storage  subsystem  that  provides  long-term 
document  storage;  fax  gateways  that  electronically  receive  and  deliver  vendor 
payment  documents  and  correspondence;  and  200  to  380  workstations, 
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consisting  of  personal  computers  (PCs)  connected  to  the  local  area  network  that 
allow  technicians  to  view  and  process  imaged  documents.  The  software  consists 
of  Wang  off-the-shelf  software,  UNIX  operating  system  software,  and  software 
applications  customized  by  the  developer  that  control  and  direct  all  EDM  system 
actions.  The  local  area  network  uses  Novell  4.1  software. 

Administration  of  Acceptance  Testing.  The  EDM  test  plan  for  system 
acceptance  was  issued  on  April  22,  1996,  and  described  test  methodology  and 
procedures  for  conducting  the  acceptance  test  and  reporting  test  results.  The 
test  plan  evaluated  functional  and  technical  requirements  for  the  document 
capture,  indexing,  and  workflow  subsystems.  For  each  evaluation  area  within  a 
subsystem,  a  test  objective,  criteria,  and  scenarios  were  developed.  The  test 
objective  stated  the  purpose  of  the  test.  The  EDM  Project  Manager  at  the 
Omaha  OPLOC  used  functional  descriptions  and  technical  documentation  to 
develop  the  criteria  as  standards  for  acceptance.  The  scenarios  described  the 
specific  methodology  to  be  used  to  conduct  the  test  and  provided  the  basis  for 
analyzing  and  recording  the  test  results. 

The  test  teams  used  standard  formats  for  problem  reports  to  document  issues 
noted  during  the  test.  The  plan  required  all  reported  problems  and  deficiencies 
to  be  categorized  in  one  of  three  ways. 

o  A  broken  designation  meant  the  problem  required  immediate  attention 
by  the  system  developer  and  a  retest  by  the  user. 

o  A  critical  designation  meant  the  problem  was  critical  to  system 
acceptance  and  required  the  concurrence  of  the  review  board  to  direct  the 
system  developer  to  implement  a  solution  before  system  acceptance. 

o  A  desirable  designation  meant  the  problem  was  not  essential  to  system 
acceptance,  but  the  review  board  would  consider  the  solution  for  future 
implementation. 

The  results  of  the  test  scenarios  and  the  problem  reports  were  used  to  assess  the 
EDM  system's  performance.  The  EDM  Project  Manager  at  the  Omaha 
OPLOC,  serving  as  the  acceptance  test  chairperson,  held  daily  meetings  with 
the  test  team  and  on-site  system  development  personnel  to  obtain  a  consensus  on 
test  results,  assign  action  items,  catalog  and  track  problem  reports,  and 
coordinate  test  activities. 

The  EDM  Project  Manager  at  the  Omaha  OPLOC  established  a  review  board 
for  EDM  system  acceptance  to  provide  corporate-level  direction  to  the  EDM 
system  acceptance  team  in  discussing  test  results  and  resolving  problem  reports. 
The  review  board  consisted  of  senior  managers  from  the  Omaha  OPLOC; 
Headquarters,  DFAS;  the  DFAS  Denver  Center;  and  the  DFAS  Financial 
Systems  Organization.  Representatives  from  the  Inspector  General,  DoD,  and 
the  Contracting  Office  Representative  from  the  DFAS  Financial  Systems 
Organization  were  advisors  to  the  board.  The  purpose  of  EDM  system 
acceptance  testing  was  to  assure  management  that  the  EDM  system  performed 
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as  intended.  The  test  results  were  to  support  the  certification  and  accreditation 
process  by  ensuring  that  security  safeguards  were  in  place  for  the  level  of  risk 
accepted  by  management. 

Status  of  Testing.  Testing  was  suspended  on  May  28,  1996,  because  of  a  high 
volume  of  documented  system  problems.  From  April  15  to  May  28,  1996, 
DFAS  conducted  system  acceptance  testing  for  the  EDM  system  prototype  at 
the  Omaha  OPLOC.  The  EDM  Project  Manager  at  the  Omaha  OPLOC  briefed 
the  EDM  system  acceptance  review  board  on  May  23,  1996,  and  recommended 
that  testing  be  suspended  while  solutions  to  the  system  problems  were  being 
implemented.  The  review  board  agreed.  DFAS  and  die  system  developer 
formally  detailed  the  system  problems  and  requirements  on  June  7,  1996.  On 
June  20,  1996,  the  system  developer  submitted  proposed  solutions  and  a 
schedule  for  testing  and  implementation.  Based  on  DFAS  approval  of  the 
solutions,  November  19,  1996,  was  established  as  the  new  acceptance  test  date. 


Discussion 

We  commend  DFAS  for  its  critical  examination  of  the  EDM  system  during  the 
testing  period,  and  for  a  detailed  accounting  of  performance  and  control 
deficiencies  identified  by  the  EDM  system  acceptance  team.  Test  results 
produced  134  open  problem  reports  that  identified  performance  and  control 
weaknesses  in  the  EDM  system.  Further,  the  134  problem  reports  identified  30 
functional  and  technical  requirements  that  needed  a  system-wide  solution  before 
acceptance  testing  could  resume.  The  EDM  system  acceptance  test  showed 
that  controls  over  the  EDM  vendor  payment  process  and  workflows  can  achieve 
management  control  objectives  related  to  the  completeness,  accuracy,  and 
authorization  of  data.  However,  improvements  are  needed  in  the  security 
controls  over  EDM  system  data.  Specifically,  controls  are  needed  to  limit 
log-on  attempts,  meet  auditability  requirements,  and  protect  secure  files. 

Results  of  EDM  System  Acceptance  Test.  The  results  of  the  EDM  system 
acceptance  test  showed  that  the  system  needed  improvements.  As  of  May  24, 
1996,  the  EDM  system  acceptance  team  prepared  307  problem  reports.  Of  the 
307  problem  reports,  the  team  determined  that  120  were  reports  of  recurring 
problems.  Of  the  remaining  187  problem  reports,  53  were  satisfactorily  closed 
mid  134  were  left  open.  After  team  analysis,  functional  and  technical  team 
leaders  designated  44  reports  as  broken,  33  reports  as  critical,  and  57  reports  as 
desirable.  A  majority  of  the  functional  problems  were  directly  or  indirectly 
related  to  technical  problems.  The  lack  of  PC  memory  and  the  system  s 
inability  to  efficiently  manage  PC  memory  were  responsible  for  at  least  49  error 
messages  that  were  recorded  during  workflow  testing.  Also,  system  response 
times  during  the  workflow  process  exceeded  acceptable  limits.  For  example, 
image  retrieval  times  averaged  26  to  33  seconds  compared  to  a  desired  time  of 
5  seconds.  Midway  through  the  testing  period,  a  team  led  by  the  Director, 
DFAS  Financial  Systems  Organization,  identified  potential  solutions  to  the 
problems  with  PC  memory  and  system  response  time.  To  ensure  that  the  EDM 
system  meets  acceptable  response  times,  a  stress  test  will  be  conducted  when 
acceptance  testing  resumes. 
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The  EDM  system  acceptance  team  and  the  system  developer  analyzed  the 
134  open  reports  and  identified  30  functional  and  technical  requirements  (21 
functional  and  9  technical  requirements)  that  needed  a  system-wide  solution 
before  EDM  system  acceptance  testing  could  resume.  Also,  according  to  the 
test  plan,  those  problem  reports  would  not  be  closed  until  the  solutions  were 
tested.  The  30  problem  areas  and  proposed  solutions  are  described  in  the 
system  developer's  draft  report,  "Solutions  A  -  T  for  the  DFAS  Electronic 
Document  Management  Partnership,"  June  20,  1996. 

Analysis  of  the  EDM  Vendor  Payment  Process  and  Workflows.  Tests 
showed  that  the  EDM  vendor  payment  process  and  the  workflow  process  can 
perform  as  described  in  the  documentation  and  can  achieve  management  control 
objectives  related  to  the  completeness,  accuracy,  and  authorization  of  data. 
During  testing,  several  functional  and  technical  problems  were  encountered  that 
required  solutions;  however,  the  system's  controls  over  the  EDM  vendor 
payment  process  and  the  seven  workflows  tested  will  perform  as  intended  when 
the  problems  are  corrected.  The  seven  workflows  are  obligation  posting  and 
invoice  posting  for  vendor  payments,  posting  of  receiving  reports,  voucher 
certification,  vouchered  for-others  processing,  disbursing  automated  payments, 
and  customer  inquiry.  Some  of  the  corrections  and  anticipated  benefits  from  the 
EDM  prototype  system  are  as  follows: 

Document  Capture  and  Indexing.  As  a  result  of  testing,  changes  to 
the  document  capture  and  indexing  subsystem  are  being  made.  Procedures  will 
be  modified  to  ensure  that  incoming  electronic  fax  documents  are  not  misrouted 
after  rebooting  the  scanning  servers.  Also,  multiple  documents  scanned 
together  require  that  the  software  allow  documents  to  be  split  and  the  document 
type  changed  while  the  same  document  is  indexed  several  times.  The  EDM 
vendor  payment  process  begins  with  the  Document  Capture  Center  (DCC), 
where  OPLOC  personnel  open,  sort,  and  batch  vendor  payment  documents. 
The  vendor  payment  documents  are  scanned  and  the  paper  documents  are 
imprinted  with  a  document  identifier  for  potential  retrieval.  The  document 
image  is  placed  in  the  electronic  indexing  queue,  along  with  electronic  fax 
images  processed  through  the  fax  monitor.  The  unages  are  indexed  by 
document  type  and  categorized  according  to  a  predetermined  workflow.  The 
EDM  subsystem  for  electronic  document  capture  provides  more  control  over 
vendor  payment  documents  than  a  manual  system  because  the  documents  are 
immediately  indexed  and  boxed,  and  are  retained  as  prescribed  by  local 
operating  procedures.  Also,  EDM  electronically  records  the  receipt  date  and 
location  of  the  document  image.  Electronic  and  manual  quality  controls  are 
used  to  ensure  that  images  are  readable,  complete,  and  accurate,  and  that 
control  logs  are  kept  for  exception  handling.  Assembling  and  batching 
documents  in  the  DCC  and  uniquely  marking  the  images  in  the  database  during 
the  indexing  process  gives  accounts  payable  technicians  and  obligation  entry 
fftphnirians  immediate  access  to  organized  sets  of  the  vendor  payment 
documentation  they  need  to  accomplish  their  work. 

Accounts  Payable  Workflows.  Acceptance  tests  identified  changes 
needed  to  make  accounts  payable  workflows  more  efficient.  For  example, 
workflow  software  will  be  modified  to  allow  the  accounts  payable  team  leader 
to  change  a  document  type,  cancel  the  old  workflow  case,  and  create  a  new  case 
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when  documents  are  indexed  incorrectly.  After  indexing,  a  workflow  case  is 
created  for  each  document  and  is  entered  into  one  of  four  accounts  payable 
workflows.  Those  workflows  are  obligation  posting  and  invoice  posting  for 
vendor  payments,  posting  of  receiving  reports,  and  customer  inquiry.  Each 
technician  performs  his  or  her  assigned  duty,  such  as  viewing  documents, 
updating  indexing  data,  researching  documents,  verifying  data,  or  entering  data 
from  document  images  into  IAPS.  Before  entering  data  into  IAPS,  technicians 
examine  the  images  for  clarity,  completeness,  and  accuracy.  The  risk  of 
entering  incorrect  data  into  IAPS  has  been  reduced.  By  using  the  Windows 
cut-and-paste  feature,  data  can  be  transferred  from  the  document  index  screen  to 
the  IAPS  screen  without  error.  Other  options  available  to  the  technicians  are 
returning  documents  to  the  DCC  for  rescanning  or  exception  handling,  and 
forwarding  documents  to  team  leaders  when  their  help  is  needed.  Management 
controls  are  enhanced  because  all  screen  actions  taken  by .  technicians  are 
recorded  and  visible  through  system  productivity  reports,  allowing  management 
to  identify  production  problems  and  evaluate  employee  performance. 

Voucher  Certification  Workflow.  As  a  result  of  testing,  several 
changes  to  the  software  for  voucher  certification  workflow  will  be  implemented 
to  make  it  more  efficient.  For  example,  the  software  will  be  modified  to  allow 
voiding  a  certified  voucher  until  the  print  file  is  sent  to  processing.  The  EDM 
system  intercepts  the  IAPS  voucher  print  files,  converts  them  to  voucher 
images,  and  creates  a  workflow  case  for  each  voucher.  The  voucher  is  inserted 
into  the  workflow  for  certification.  During  this  process,  supporting  document 
images  are  added  to  each  workflow  case  by  linking  contract  numbers,  invoice 
numbers,  and  dates  when  goods  and  services  are  received.  Certifying  officers 
view  the  workflow  cases  containing  the  voucher  and  supporting  document 
images,  and  certify,  void,  or  return  the  voucher  folder  to  an  accounts  payable 
technician  for  additional  review.  This  workflow  has  the  greatest  potential  for 
saving  time,  and  the  automatic  assembly  of  supporting  documents  for  each 
voucher  has  the  greatest  potential  for  increasing  productivity. 

Disbursing  and  For-Others  Workflows.  The  workflow  for  disbursing 
automated  payments  needed  correction  to  show  separate  totals  for  cash,  check, 
and  electronic  funds  transfer  payments  in  the  reconciliation  report,  while  no 
corrections  were  needed  to  the  for-others  workflow.  The  for-others  and 
disbursing  automated  payment  workflows  are  initiated  upon  voucher 
certification.  The  workflow  for  vouchered  for-others  processing  recognizes 
certified  vouchers  marked  "for-others  processing"  and  routes  the  voucher 
package  to  the  assembly  technician,  who  prints  the  documents  and  forwards 
them  to  the  appropriate  accountable  station.  The  workflow  for  disbursing 
automated  payments  makes  voucher  status  data  available  to  the  disbursing 
section,  where  personnel  reconcile  voided  and  certified  vouchers  between  IAPS 
and  EDM.  The  disbursing  section  electronically  forwards  the  reconciled  and 
certified  check  print  files  to  the  DFAS  Denver  Center  for  payment. 

Analysis  of  Security  Requirements  for  EDM  Systems.  Security  controls  over 
EDM  system  data  needs  improvement.  The  results  of  EDM  system  acceptance 
testing  showed  that  the  system  could  not  demonstrate  the  necessary  log-on 
security,  audit  trails,  and  safeguards  for  protecting  secure  files.  According  to 
DoD  Directive  5200.28,  "Security  Requirements  for  Automated  Information 
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Systems  (AIS),"  March  21,  1988,  DoD  automated  information  systems  require 
a  Class  C2  level  of  security  (Controlled  Access  Protection).  The  C2  standard  is 
defined  in  DoD  5200.28  Standard,  "Trusted  Computer  System  Evaluation 
Criteria,"  December  1985.  Systems  in  this  class  use  access  control  to  protect 
files  and  programs  from  unauthorized  access.  Users  are  made  individually 
accountable  for  their  actions  through  log-on  procedures  and  auditing  of 
security-related  events  such  as  users  exceeding  limits  for  log-on  attempts. 
Logging  functions  that  collect  data  on  security-related  events  should  also  be 
isolated  from  the  data  they  protect,  and  should  be  subject  to  the  same  access 
control  and  auditing  requirements. 

Access  Control.  The  system  administrator  exercises  access  control  by 
providing  individuals  with  access  to  the  EDM  system  in  accordance  with  access 
lists  approved  by  management.  The  lists  identify  the  role  assignments  for  which 
each  individual  is  authorized.  The  role  assignments  are  defined  by  system 
parameters  and  linked  to  user  identification  numbers  set  by  the  system 
administrator.  The  EDM  system  authenticates  the  user  identification  number  by 
matching  it  to  a  password  recognized  by  the  system  and  known  only  by  the 
user.  In  this  manner,  manual  and  automated  access  controls  ensure  that  vital 
workflows  necessary  to  process  vendor  payment  documentation  are  limited  to 
technicians  who  are  authorized  to  do  the  work.  The  system  developer  is 
responsible  for  system  administrator  functions  until  the  function  is  turned  over 
to  OPLOC  personnel. 

Log-on  Procedures.  EDM  applications  and  the  UNIX  operating  system 
allowed  unlimited  attempts  to  log  on  to  the  system.  The  system  administrator 
determines  the  number  of  attempts  allowed.  The  generally  accepted  standard 
for  automated  information  systems  is  to  allow  three  attempts,  after  which  the 
system  administrator  must  intervene.  The  risk  in  allowing  unlimited  attempts  is 
that  an  individual  could  eventually  guess  the  user's  password  and  gain  access  to 
the  system  without  the  system  administrator's  knowledge.  The  system 
developer  has  developed  a  solution  for  the  EDM  applications,  but  not  for  the 
UNIX  operating  system. 

Audit  Trails.  Features  of  off-the-shelf  software  designed  to  ensure 
aHpqnatP  audit  trails  were  not  turned  on  during  acceptance  testing.  Therefore, 
the  test  team  could  not  determine  to  what  extent  operating  system  events  and 
application  system  events  could  be  reconstructed,  or  how  system  performance 
would  have  been  affected  if  the  features  had  been  turned  on.  In  response  to 
problem  reports  on  die  adequacy  of  EDM  audit  trails,  the  system  developer 
maHp  the  following  statement  in  the  "Omaha  Detailed  Requirements  for  the 
DFAS  Electronic  Document  Management  Partnership,"  June  7,  1996. 

A  substantial  amount  of  raw  information  is  currently  being  gathered 
by  some  of  the  application  software  products  being  used  in  the  EDM 
system.  These  applications  include  die  Oracle  database  and  the  Wang 
OPEN/workflow  database.  In  some  cases  software  tools  exist  to 
extract  and  format  this  data  for  auditing  purposes.  In  many  cases  new 
software  tools  will  have  to  be  developed.  New  development  will  be 
required  to  gather  and  report  the  other  data  events  which  occur  in  the 
EDM  system.  In  the  operating  system  area  [UNIX],  very  little 
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information  is  currently  being  gathered  and  the  operating  system 
software  provides  little  capability  to  capture  the  information  without 
extensive  operating  system  modification. 

At  a  minimum,  system  transaction  and  access  logs  should  be  date-  and  time- 
stamped  for  each  individual  user  session.  Audit  data  must  also  be  available  for 
reconstruction  of  any  user  session  to  aid  security  review  or  auditing.  The 
system  developer  stated  that  transaction  logging  data  could  be  collected  if  the 
portion  of  the  EDM  system  that  runs  on  Hewlett-Packard  computers  could  run 
under  the  Hewlett-Packard  "Trusted  System".  However,  according  to  the 
system  developer,  that  trusted  system  is  not  compatible  with  the  Network 
Information  Service  configuration  used  by  EDM  to  provide  basic  user 
authentication  for  the  Wang  products.  We  are  working  with  the  DFAS 
Financial  Systems  Organization,  the  EDM  Project  Manager  at  the  Omaha 
OPLOC,  and  the  system  developer  to  define  a  format  for  creating  an  adequate 
audit  trail. 

Secure  File  Protection.  The  password  file  in  UNIX  is  not  hidden  from 
potential  unauthorized  system  users  to  ensure  file  protection.  Once  access  to 
UNIX  is  obtained,  individuals  can  read  the  list  of  user  identifications  in  the 
secure  password  file  and  attempt  to  decode  the  encrypted  passwords.  System 
integrity  would  be  compromised  if  an  unauthorized  user  gained  access  to  the 
system's  production,  logging,  or  program  data,  because  no  record  would  be 
made  of  the  files  accessed  or  actions  taken  against  those  files.  In  addition,  an 
unauthorized  user  could  alter  the  audit  files,  thus  eliminating  any  evidence  of 
intrusion.  The  system  developer  and  personnel  from  the  DFAS  Mid-Tier 
Maintenance  Organization  have  discussed  the  problem  and  proposed  several 
approaches  to  making  the  password  file  less  visible  and  reducing  the  risk  that 
unauthorized  users  can  access  EDM  system  files. 

Other  preventive  controls  are  in  place  to  protect  system  data  from  unauthorized 
users.  For  example,  the  first  log-on  required  to  access  the  EDM  system 
applications  is  through  the  Novell  4.1  network  software,  which  limits  log-on 
attempts.  Also,  a  barrier  to  unauthorized  entry  over  the  internet  is  provided  by 
the  Transmission  Control  Protocol  Wrapper  Program,  which  requires  positive 
identification  by  internet  protocol  address  of  all  attempts  to  access  the  EDM 
system. 


Summary 

EDM  system  acceptance  testing  produced  134  open  problem  reports  that 
identified  system  performance  and  control  weaknesses.  Those  134  problem 
reports  identified  30  functional  and  technical  requirements  that  needed  a 
system-wide  solution  before  acceptance  testing  could  resume.  The  EDM  system 
acceptance  test  showed  that  the  controls  over  the  EDM  system  s  vendor 
payment  process  and  workflows  can  achieve  management  control  objectives 
related  to  the  completeness,  accuracy,  and  authorization  of  data,  and  can 
perform  as  intended  when  the  30  identified  requirements  are  satisfied.  Included 
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in  those  requirements  is  a  need  for  DFAS  to  improve  the  EDM  system's 
security  controls  in  order  to  limit  log-on  attempts,  meet  system  auditability 
requirements,  and  protect  secure  files.  DFAS  initiated  corrective  action, 
therefore,  we  are  not  making  any  recommendations  at  this  time.  We  will 
continue  to  review  the  EDM  system  in  the  next  phase  of  our  evaluation. 


Management  Comments 

We  provided  DFAS  a  draft  of  this  report  on  September  9,  1996,  for  review  and 
comments.  We  did  not  request  written  management  comments,  and  none  were 
provided.  We  discussed  the  draft  report  and  the  results  of  our  evaluation  with 
the  EDM  Project  Management  Office.  The  EDM  Project  Office  agreed  with 
the  results  of  our  evaluation  and  suggested  minor  changes  to  the  final  report. 
We  agreed  that  those  changes  were  appropriate  and  included  them  in  the  final 
report. 

We  appreciate  the  courtesies  extended  to  the  evaluation  staff.  If  you  have  any 
questions  or  wish  to  discuss  this  report,  please  contact  Mr.  Christian  Hendricks, 
Evaluation  Program  Director,  at  (703)  604-9139,  or  Mr.  Carl  Zielke, 
Evaluation  Project  Manager,  at  (703)  604-9147.  The  distribution  of  this  report 
is  listed  in  Enclosure  2.  Evaluation  team  members  are  listed  inside  the  back 
cover. 

David  K.  Steensma 
Deputy  Assistant  Inspector  General 
for  Auditing 
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